Deep combination of radar with optical data for gesture recognition: role of attention in fusion architectures

Multimodal time series classification is an important aspect of human gesture recognition, in which limitations of individual sensors can be overcome by combining data from multiple modalities. In a deep learning pipeline, the attention mechanism further allows for a selective, contextual concentration on relevant features. However, while the standard attention mechanism is an effective tool when working with Natural Language Processing (NLP), it is not ideal when working with temporally- or spatially-sparse multi-modal data. In this paper, we present a novel attention mechanism, Multi-Modal Attention Preconditioning (MMAP). We first demonstrate that MMAP outperforms regular attention for the task of classification of modalities involving temporal and spatial sparsity and secondly investigate the impact of attention in the fusion of radar and optical data for gesture recognition via three specific modalities: dense spatiotemporal optical data, spatially sparse/temporally dense kinematic data, and sparse spatiotemporal radar data. We explore the effect of attention on early, intermediate, and late fusion architectures and compare eight different pipelines in terms of accuracy and their ability to preserve detection accuracy when modalities are missing. Results highlight fundamental differences between late and intermediate attention mechanisms in respect to the fusion of radar and optical data

Digital twins: a survey on enabling technologies, challenges, trends and future prospects

Digital Twin (DT) is an emerging technology surrounded by many promises, and potentials to reshape the future of industries and society overall. A DT is a system-of-systems which goes far beyond the traditional computer-based simulations and analysis. It is a replication of all the elements, processes, dynamics, and firmware of a physical system into a digital counterpart. The two systems (physical and digital) exist side by side, sharing all the inputs and operations using real-time data communications and information transfer. With the incorporation of Internet of Things (IoT), Artificial Intelligence (AI), 3D models, next generation mobile communications (5G/6G), Augmented Reality (AR), Virtual Reality (VR), distributed computing, Transfer Learning (TL), and electronic sensors, the digital/virtual counterpart of the real-world system is able to provide seamless monitoring, analysis, evaluation and predictions. The DT offers a platform for the testing and analysing of complex systems, which would be impossible in traditional simulations and modular evaluations. However, the development of this technology faces many challenges including the complexities in effective communication and data accumulation, data unavailability to train Machine Learning (ML) models, lack of processing power to support high fidelity twins, the high need for interdisciplinary collaboration, and the absence of standardized development methodologies and validation measures. Being in the early stages of development, DTs lack sufficient documentation. In this context, this survey paper aims to cover the important aspects in realization of the technology. The key enabling technologies, challenges and prospects of DTs are highlighted. The paper provides a deep insight into the technology, lists design goals and objectives, highlights design challenges and limitations across industries, discusses research and commercial developments, provides its applications and use cases, offers case studies in industry, infrastructure and healthcare, lists main service providers and stakeholders, and covers developments to date, as well as viable research dimensions for future developments in DTs

Challenges ahead in cyber physical systems: a coding survey

This paper is directed towards discussing the challenges associated with cyber physical Systems (CPSs) as they encompass a broad collection of components integrating cyberspace and mechanical elements together. For a better understanding of the challenges associated with CPS, the paper provides a detailed foundation of CPSs including definitions of the various elements. Subsequently, we provide a qualitative content analysis of the current research in the field demonstrating issues related to the technical development, economic policies and the effect of those on the design phase of a CPS. Finally, we provide several recommendations to improve the design and implementation of CPS

Augmented-reality computer-vision assisted disaggregated energy monitoring and IoT control platform

The aim of this research is to develop an innovative low cost and affordable platform for smart home control and energy monitoring interfaced with augmented reality. This method will educate people about energy use at a time when fuel costs are rising and create novel methods of interaction for those with disabilities. In order to increase the awareness of energy consumption, we have developed an interactive system using Augmented Reality to show live energy usage of electrical components. This system allows the user to view his real time energy consumption and at the same time offers the possibility to interact with the device in Augmented Reality. The energy usage was captured and stored in a database which can be accessed for energy monitoring. We believe that the combinations of both, complex smart home applications and transparent interactive user interface will increase the awareness of energy consumption

Air quality monitoring in Mauritius

The project aims at investigating the ambient air quality in Mauritius. The proposed system will use Internet of Things (IoT) devices equipped with gas sensors to capture the level of pollutant in the air at various locations around the island. The monitoring devices consist of low cost devices such as Raspberry Pi 3 and Arduino boards. The main pollutants being monitored are Ozone (O3), Carbon Monoxide (CO), Oxides of Nitrogen (NOx) and Particulate Matter (PM2.5 and PM10). Additional environmental factors such as temperature, atmospheric pressure and relative humidity are also measured. The sensor data along with information such as the identifier of the monitoring system, time-stamp for the data captured, GPS coordinates of the sensing location are captured at regular interval during the data and sent to the Microsoft Azure platform

Challenges ahead in cyber physical systems: a coding survey

This paper is directed towards discussing the challenges associated with cyber physical Systems (CPSs) as they encompass a broad collection of components integrating cyberspace and mechanical elements together. For a better understanding of the challenges associated with CPS, the paper provides a detailed foundation of CPSs including definitions of the various elements. Subsequently, we provide a qualitative content analysis of the current research in the field demonstrating issues related to the technical development, economic policies and the effect of those on the design phase of a CPS. Finally, we provide several recommendations to improve the design and implementation of CPS

Air quality monitoring in Mauritius

The project aims at investigating the ambient air quality in Mauritius. The proposed system will use Internet of Things (IoT) devices equipped with gas sensors to capture the level of pollutant in the air at various locations around the island. The monitoring devices consist of low cost devices such as Raspberry Pi 3 and Arduino boards. The main pollutants being monitored are Ozone (O3), Carbon Monoxide (CO), Oxides of Nitrogen (NOx) and Particulate Matter (PM2.5 and PM10). Additional environmental factors such as temperature, atmospheric pressure and relative humidity are also measured. The sensor data along with information such as the identifier of the monitoring system, time-stamp for the data captured, GPS coordinates of the sensing location are captured at regular interval during the data and sent to the Microsoft Azure platform